Apparatus and method for generating 3d printing model using multiple textures

ABSTRACT

An apparatus for generating a 3D printing model using multiple textures includes a template model storage unit configured to store template models that are 3D models previously manufactured to be printable, a 3D model conversion unit configured to, upon receiving a 3D input model and a texture image, perform mesh deformation on the template model stored in the template model storage unit by using geometric information about the 3D input model, a texture processing unit configured to assign the texture image to the 3D model having being subjected to the mesh deformation, and a 3D printing model output unit configured to output a geometric model and a texture image of the template model which are finally calculated.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to Korean Patent Application No.10-2016-0025789, filed on Mar. 3, 2016, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference in its entirety.

BACKGROUND

1. Field

The following description relates to a three dimensional (3D) printingtechnology, and more particularly, to an apparatus and a method forgenerating a 3D model.

2. Description of Related Art

In the recent years, 3D printing technologies are drawing attention, andcontent when using a 3D printer becomes more and more important. As fora method of generating 3D content, a mobile device equipped with a depthsensor, a stereo camera, etc. or a handheld 3D scanner using a laser,visible rays is widely used while serving important roles.

Also, there is an increasing use for a method in which surroundings ofan object are imaged using a single camera and a 3D model is generatedbased on the images and a method in which a plurality of cameras areinstalled and a model is imaged by the plurality of cameras at one time.

However, with the 3D content generating method diversified, variousdevices and apparatuses use different geometric informationreconstruction methods, different texture forms, and different outputfile formats. In addition, scan data having a great amount of errors andbeing heavy is not easy to use directly for a 3D printing, and thus apost-processing is needed. In particular, a texture, a major factor fordetermining the quality of a final printout of 3D printing, is processedin different methods depending on devices, but the method is not evenopen to the public, so that there is difficulty in the post processing.

As for most existing 3D model-based texture processing methods, assumingthat there is given a 3D model and a texture image which matches the 3Dmodel, a correspondence between a 3D model and a texture image is found,and texture mapping is performed using the correspondence. Such a methodis applicable to the traditional scanning technique in which a 3D modelis reconstructed, an image is captured in front of the 3D model, and theimage is assigned to texture, but is not applicable to textures used inthe currently emerging 3D model generating methods, for example,multiple textures, texture segments, and randomly located textures.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

The following description relates to an apparatus and a method forgenerating a 3D printing model using multiple textures, capable ofgenerating a final 3D model usable with 3D printing regardless of theform of a texture, including not only multiple textures but also variousother types of textures usable in the current 3D model generatingschemes, such as a mobile device and a hand held scanner.

In one general aspect, an apparatus for generating a 3D printing modelusing multiple textures includes a template model storage unit, a 3Dmodel conversion unit, a texture processing unit, and a 3D printingmodel output unit. The template model storage unit may be configured tostore template models that are 3D models previously manufactured to beprintable. The 3D model conversion unit may be configured to, uponreceiving a 3D input model and a texture image, perform mesh deformationon the template model stored in the template model storage unit by usinggeometric information about the 3D input model. The texture processingunit may be configured to assign the texture image to the 3D modelhaving being subjected to the deformation. The 3D printing model outputunit may be configured to output a geometric model and a texture imageof the template model which are finally calculated.

The correspondence setting unit may be configured to calculate acorrespondence between the 3D input model and the template model havingfinished the mesh deformation, and search for a triangle of the deformedmodel which corresponds to each vertex of the 3D input model. Thevalidity check unit may be configured to compare a reconstruction areaof the 3D input model with a reconstruction area of the template modelto calculate a part of texture areas of the template model to which atexture needs to be newly assigned. The texture warping unit may beconfigured to warp a triangle of the input texture image into a triangleof a texture of the template model with respect to a valid area based onthe set correspondence. The color matching unit may be configured totransform a remaining part of the texture areas of the template model tohave a same color as a color of the texture of the input model.

The validity check unit may set a minimum area in consideration of aviewing direction of a user and a normal direction of a model.

The texture warping unit may calculate a vertex value throughinterpolation with respect to inside of a triangle of the template modelcorresponding to the input 3D model.

In another general aspect, a method for generating a 3D printing modelusing multiple textures includes: receiving a 3D input model and atexture image; performing mesh deformation on a template model that is a3D model previously manufactured to be printable, by using geometricinformation about the 3D input model; assigning the texture image to the3D model having being subjected to the mesh deformation; and outputtinga geometric model and a texture image of the template model which arefinally calculated.

The assigning of the texture image may include: calculating acorrespondence between the 3D input model and the template model havingcompleted the mesh deformation, and searching for a triangle of thedeformed model which corresponds to each vertex of the 3D input model;calculating a part of texture areas of the template model to which atexture needs to be newly assigned by comparing a reconstruction area ofthe 3D input model with a reconstruction area of the template model;warping a triangle of the input texture image into a triangle of atexture of the template model with respect to a valid area based on theset correspondence; and transforming a remaining part of the textureareas of the template model to have a same color as a color of thetexture of the input model.

The calculating of a part of texture areas of the template model towhich a texture needs to be newly assigned may include setting a minimumarea in consideration of a viewing direction of a user and a normaldirection of a model.

The warping of a triangle of the input texture image into a triangle ofa texture of the template model may include calculating a vertex valuethrough interpolation with respect to inside of a triangle of thetemplate model corresponding to the input 3D model.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an apparatusfor generating a 3D printing model using multiple textures according anembodiment of the present invention.

FIGS. 2A and 2B are diagrams illustrating examples of a texture image.

FIG. 3 is a detailed block diagram illustrating a texture processingunit according to an embodiment of the present invention.

FIG. 4 is a flowchart for describing a method for generating a 3Dprinting model using multiple textures according to an embodiment of thepresent invention.

FIG. 5 is a flowchart for describing a texture processing operationaccording to an embodiment of the present invention.

Throughout the drawings and the detailed description, unless otherwisedescribed, the same drawing reference numerals will be understood torefer to the same elements, features, and structures. The relative sizeand depiction of these elements may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention will be described indetail below with reference to the accompanying drawings to assist thereader in gaining a comprehensive understanding of the methods,apparatuses, and/or systems described herein.

In describing the present invention, detailed descriptions that arewell-known but are likely to make the subject matter of the presentinvention unclear will be omitted in order to avoid redundancy.

The terminology used herein is defined in consideration of its functionin the present invention, and may vary with an intention of a user andan operator or custom. Accordingly, the definition of the terms shouldbe determined based on overall contents of the specification.

FIG. 1 is a block diagram illustrating a configuration of an apparatusfor generating a 3D printing model using multiple textures according anembodiment of the present invention.

Referring to FIG. 1, an apparatus for generating a 3D printing modelusing multiple textures (hereinafter, referred to as ‘an apparatus’)includes a template model storage unit 110, a 3D model conversion unit120, a texture processing unit 130, and a 3D printing model output unit140.

The template model storage unit 110 stores template models that are 3Dmodels previously manufactured to be printable.

The 3D model conversion unit 120, upon receiving a 3D input model and atexture image, performs mesh deformation on the template model stored inthe template model storage unit 110 by using geometric information aboutthe 3D input model. That is, a template model configured for 3D printingis transformed to a 3D input model. The template model is a templatemodel having an appearance maximally similar to that of the input 3Dmodel and manufactured in consideration of characteristics of 3Dprinting, and thus is suitable for 3D printing. In addition, the input3D model and the input texture image may represent a 3D model and atexture image that are obtained by any one of various 3D modelgenerating methods.

One of the most important criteria for determining the degree oftransformation is texture information. However, unless a limitation isput on an input scheme, the texture processing method may vary with theform of an input texture image. The conventional technology is largelydivided into a texture processing method used when a texture image and a3D geometric model are given without information about a camera and atexture processing method used when information about a camera is givenand a correspondence between a texture image and a geometric model iscalculated and then texture mapping is performed. In other words, thesesmethods are suggested by assuming an input texture image that representsan appearance and a 3D model that corresponds to the texture image, andvarious approaches are present to increase the accuracy of the mapping,for example, parameterization, remeshing, and so on. However, when aninput texture image is provided in the form of multiple texturescalculated from a plurality of pictures or is obtained using respectiveown texture acquisition methods of individual devices, a texture imageis acquired in a totally different form from a texture image that hasbeen conventionally input and used. FIGS. 2A and 2B are diagramsillustrating examples of a texture image that are different from theconventional texture images. The conventional texture processing methodhave difficulty in assigning the texture images shown in FIGS. 2A and 2Bto a texture of a template model, and thus input texture image data issubject to limitation.

According to the present application, the texture processing unit 130assigns to the mesh deformationed 3D model the texture image that has arandom form rather than a consistent form used in the conventionaltechnology. Details thereof will be described below with reference toFIGS. 2 and 3.

The 3D printing model output unit 140 outputs a geometric model and atexture image of the template model which are finally calculated.

FIG. 3 is a detailed block diagram illustrating a texture processingunit according to an embodiment of the present invention.

Referring to FIG. 3, the texture processing unit 130 includes acorrespondence setting unit 131, a validity check unit 132, a texturewarping unit 133, and a color matching unit 134.

The correspondence setting unit 131 calculates a correspondence betweenthe 3D input model and the template model having finished the meshdeformation, and searches for a triangle of the deformed model whichcorresponds to each vertex of the 3D input model.

The validity check unit 132 compares a reconstruction area of the 3Dinput model with a reconstruction area of the template model tocalculate a part of texture areas of the template model to which atexture needs to be newly assigned. According to an embodiment of thepresent invention, a minimum area is set in consideration of a viewingdirection of a user and a normal direction of a model.

The texture warping unit 133 warps a triangle of the input texture imageinto a triangle of a texture of the template model with respect to avalid area based on the set correspondence. According to an embodimentof the present invention, considering that an input model is generallyheavy and includes a number of triangles, vertex values are calculatedby performing interpolation with respect to inside of a triangle of thetemplate model corresponding to the input model.

That is, upon the mesh deformation of the template model, the templatemodel is arranged at the same position as that of the input model, andis transformed to have an appearance maximally similar to that of theinput model. Thereafter, through calculation of a correspondence, thetexture of the input model is deformed to the template model.

The color matching unit 134 transforms a remaining part of the textureareas of the template model to have a color identical to a color of thetexture of the input model. Since the texture of the input model isassigned to only a part of the texture areas of the template model, thecolor matching unit 134 changes the remaining part to have a coloridentical to that of the texture of the 3D input model.

FIG. 4 is a flowchart for describing a method for generating a 3Dprinting model using multiple textures according to an embodiment of thepresent invention.

Referring to FIG. 4, an apparatus, upon receiving a 3D input model and atexture image (S410), detects a template model which is a 3D modelpreviously manufactured to be printable and having an appearance similarto that of the 3D input model (S420). Then, the apparatus performs meshdeformation on the detected template model by using geometricinformation about the 3D input model (S430). That is, a template modelconfigured for 3D printing is transformed into a 3D input model. Thetemplate model is a template model which has an appearance maximallysimilar to an input 3D model and manufactured in consideration ofcharacteristics of 3D printing, and thus is suitable for 3D printing.The input 3D input model and the input texture image may be a 3D modeland a texture image that are obtained by any one of various 3D modelgenerating methods.

Then, the apparatus assigns to the mesh deformationed 3D model thetexture image that has a random form rather than a consistent form usedin the conventional technology (S440). Details thereof will be describedbelow with reference to FIG. 5.

The apparatus outputs a geometric model and a texture image which arefinally calculated (S450).

FIG. 5 is a flowchart for describing a texture processing operationaccording to an embodiment of the present invention.

Referring to FIG. 5, the apparatus calculates a correspondence betweenthe 3D input model and the template model having finished the meshdeformation, and searches for a triangle of the deformed model whichcorresponds to each vertex of the 3D input model (S510).

Then, the apparatus compares a reconstruction area of the 3D input modelwith a reconstruction area of the template model, and calculates a partof texture areas of the template model to which a texture needs to benewly assigned (S520). According to an embodiment of the presentinvention, a minimum area is set in consideration of a viewing directionof a user and a normal direction of a model.

Then, the apparatus warps a triangle of the input texture image into atriangle of a texture of the template model with respect to a valid areabased on the set correspondence (S530). According to an embodiment ofthe present invention, considering that an input model is generallyheavier and has a number of triangles, vertex values are calculated byperforming interpolation with respect to inside of a triangle of thetemplate model corresponding to the input model. That is, upon the meshdeformation of the template model, the template model is arranged at thesame position as that of the input mode and is transformed to have anappearance maximally similar to that of the input model. Thereafter, theapparatus performs an operation of transferring the texture of the inputmodel to the template model by calculating a correspondence.

The apparatus transforms a remaining part of the texture areas of thetemplate model to have a color identical to a color of the texture ofthe input model (S540). In other words, the texture of the input modelis assigned to only some part of the texture areas of the templatemodel, and the apparatus changes the remaining part of the texture areasof the template model to have the same color as that of the texture ofthe 3D input model.

As is apparent from the above, when an input texture image is providedas a multiple texture calculated from a plurality of pictures or isobtained using respective own texture acquisition schemes of various 3Dmodel acquisition apparatuses or systems, a texture image is output in aform that is totally different from a texture image having been used asan input in the existing technology. According to the existing approach,such a texture image is not easily assigned to a texture of a templatemodel, and thus data of an input unit has limitations.

As to remove the above described constraints, the present inventionallows newly emerging various 3D model acquisition schemes to be adoptedin many ways and used in the 3D printing. Accordingly, useraccessibility and data compatibility with regard to generating 3Dprinting content can be remarkably improved.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the invention as definedby the appended claims. Therefore, the preferred embodiments should beconsidered in descriptive sense only and not for purposes of limitation,and also the technical scope of the invention is not limited to theembodiments. Furthermore, the present invention is defined not by thedetailed description of the invention but by the appended claims, andall differences within the scope will be construed as being comprised inthe present disclosure.

What is claimed is:
 1. An apparatus for generating a 3D printing modelusing multiple textures, the apparatus comprising: a template modelstorage unit configured to store template models that are 3D modelspreviously manufactured to be printable; a 3D model conversion unitconfigured to, upon receiving a 3D input model and a texture image,perform mesh deformation on the template model stored in the templatemodel storage unit by using geometric information about the 3D inputmodel; a texture processing unit configured to assign the texture imageto the 3D model having being subjected to the mesh deformation; and a 3Dprinting model output unit configured to output a geometric model and atexture image of the template model which are finally calculated.
 2. Theapparatus of claim 1, wherein the texture processing unit comprises: acorrespondence setting unit configured to calculate a correspondencebetween the 3D input model and the template model having finished themesh deformation, and search for a triangle of the deformed model whichcorresponds to each vertex of the 3D input model; a validity check unitconfigured to compare a reconstruction area of the 3D input model with areconstruction area of the template model to calculate a part of textureareas of the template model to which a texture needs to be newlyassigned; a texture warping unit configured to warp a triangle of theinput texture image into a triangle of a texture of the template modelwith respect to a valid area based on the set correspondence; and acolor matching unit configured to transform a remaining part of thetexture areas of the template model to have a same color as a color ofthe texture of the input model.
 3. The apparatus of claim 2, wherein thevalidity check unit sets a minimum area in consideration of a viewingdirection of a user and a normal direction of a model.
 4. The apparatusof claim 2, wherein the texture warping unit calculates a vertex valuethrough interpolation with respect to inside of a triangle of thetemplate model corresponding to the input 3D model.
 5. A method forgenerating a 3D printing model using multiple textures, the methodcomprising: receiving a 3D input model and a texture image; performingmesh deformation on a template model that is a 3D model previouslymanufactured to be printable, by using geometric information about the3D input model; assigning the texture image to the 3D model having beingsubjected to the mesh deformation; and outputting a geometric model anda texture image of the template model which are finally calculated. 6.The method of claim 5, wherein the assigning of the texture imagecomprises: calculating a correspondence between the 3D input model andthe template model having completed the mesh deformation, and searchingfor a triangle of the deformed model which corresponds to each vertex ofthe 3D input model; calculating a part of texture areas of the templatemodel to which a texture needs to be newly assigned by comparing areconstruction area of the 3D input model with a reconstruction area ofthe template model; warping a triangle of the input texture image into atriangle of a texture of the template model with respect to a valid areabased on the set correspondence; and transforming a remaining part ofthe texture areas of the template model to have a same color as a colorof the texture of the input model.
 7. The method of claim 6, wherein thecalculating of a part of texture areas of the template model to which atexture needs to be newly assigned includes setting a minimum area inconsideration of a viewing direction of a user and a normal direction ofa model.
 8. The method of claim 6, wherein the warping of a triangle ofthe input texture image into a triangle of a texture of the templatemodel includes calculating a vertex value through interpolation withrespect to inside of a triangle of the template model corresponding tothe input 3D model.